{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "#r \"nuget: TorchSharp-cpu\"\n",
    "\n",
    "using TorchSharp;\n",
    "using static TorchSharp.TensorExtensionMethods;"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensors\n",
    "\n",
    "In TorchSharp, as in all deep learning, the fundamental data type is a 'tensor,' which is simply a generalized matrix. In Linear Algebra, a one-dimensional arrays are called 'vector,' and a two-dimensional array is a 'matrix.' Generalizing on that, a tensor is simply an N-dimensional array. \n",
    "\n",
    "Please note that there is an overloaded use of the word 'dimensions' here -- in physics, a vector (one dimension) with three elements is used to represents a point in space, one element for each spatial dimension. When we speak of 'dimension' in these tutorials, it is the number of tensor dimensions that is of interest.\n",
    "\n",
    "So, let's get started with tensors by creating some."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constant-Filled Tensors\n",
    "\n",
    "The simplest tensor creation primitives just initializes a tensor with either 0 or 1 in all its elements. The arguments passed in is the size of each dimension. Think of the first dimension as the rows of a table, the second as the columns, and then you just have to generalize things in your head after that. In the examples below, we'll mostly be creating 3x4 matrices, for simplicity.\n",
    "\n",
    "One thing to note is that .NET Interactive will show the object and its fields, etc. when you say just 't' at the end of the notebook cell. What we want for tensors is to show the contents, and there's a special version of ToString() taking a Boolean that shows not just the size and type of the tensor, but also its contents. The special .NET Interactive formatter that is found at the top of the notebook (and should be at the top of all notebooks using TorchSharp) uses TorString(true)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "var t = torch.ones(3,4);\n",
    "t"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have more than two dimensions, the special version of ToString(t) will try to format it in a way that makes sense to a human:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.ones(2,4,4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you intend to fill the tensor with values from somewhere else, in other words because you pre-allocated it, there's an 'empty' factory that is faster than using anything else. The values are just whatever was found in memory when the tensor was created. Don't mistake that for random values, though -- they won't adhere to any particular distribution. In fact, one of the main uses for empty tensors is to create one, and then fill it with random numbers from some particular distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.empty(4,4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also create a tensor from any value you want:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.full(4,4,3.14f)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, you want to display more than one value in a cell. To do that, you need to rely on good-old-fashioned printing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "var style = torch.TensorStringStyle;\n",
    "Console.WriteLine(torch.zeros(4,4).ToString(style));\n",
    "Console.Write(torch.ones(4,4).ToString(style));"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All that typing will get pretty tedious, so we added an extension method `print()` that is much quicker to type. It really should only be used in notebooks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.zeros(4,4).print();\n",
    "torch.ones(4,4).print();"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may have noticed that each tensor has a 'type = Float32' attribute. This is a peculiarity about the TorchSharp tensor type -- the element type does not show up in the type, Tensor is not Tensor\\<T\\>. This is so because the underlying C++ / CUDA runtime represents tensors this way, and it makes it easier to port code from Python, too.\n",
    "\n",
    "You may also have noticed that tensors are created using factories, not constructors. Also, the naming convention doesn't look anything like .NET. We chose to step away from .NET conventions in order to make it easier to port code from Python. We know this will upset some, and please some, but it's the decision we came to after a long time of deliberating.\n",
    "\n",
    "Anyway, 'Float32' is the default, but you can create tensors of other types, too, including complex tensors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.zeros(4,15, dtype: torch.int32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t = torch.zeros(4,4, dtype: torch.complex64)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To access the contents of a tensor, you treat it as a multi-dimensional array (note that the number of dimensions also isn't part of the type itself). When you do, you'll be surprised to see that the result of the indexing operator is another tensor, one that has no shape -- this is how TorchSharp represents a scalar value. Later in this tutorial, we will see why. For now, just know that you have to extract the value using a function, based on the type you expect to get out.\n",
    "\n",
    "In PyTorch, there's a method '.item()' used for this purpose. In TorchSharp, it's a templatized method: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t = torch.zeros(4,4, dtype: torch.int32);\n",
    "Console.Write(t[0,0]);\n",
    "t[0,0].item<int>()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To write to a single element, you don't have to do anything special to create a tensor from the value you want to write."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t[0,0] = 35;\n",
    "t"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Randomized Tensors\n",
    "\n",
    "In machine learning, random number generation is very important, and you often end up using the RNG APIs to create tensors. There are a big number of RNGs, most of them for floating point values, but there are some for integers, too.\n",
    "\n",
    "The usual suspects are present -- normal and uniform distributions, binomial (true/false or 0/1) and uniformly distributed integers. A separate tutorial will disscuss random number generation in more detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Normal distribution\n",
    "torch.randn(3,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Uniform distribution between [0,1[\n",
    "torch.rand(3,4)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To change the range, just multiply and/or add:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Uniform distribution between [100,110[\n",
    "(torch.rand(3,4) * 10 + 100)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main factory for integer values is not quite as convenient to use -- in the function signature, there is an integer to pass in for the max value, so the dimension values have to be passed in an array or tuple. In C#, the tuple literal syntax is more concise than array literals:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.randint(10, (3,4))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's a lot more to know about random number generation -- TorchSharp offers a great many distributions and way to initialize tensor. This space is so rich, it has its own tutorial coming up later."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensors and .NET Arrays\n",
    "\n",
    "Tensors are needed when using TorchSharp operations, providing the right abstractions for parallelism and GPU acceleration. However, it is sometimes necessary to either create a tensor from .NET arrays, or get a .NET array out of a tensor. If for no other reason, to do I/O.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Creating a Tensor\n",
    "\n",
    "Several tensor factories exist that take .NET arrays as input. The main difference is whether they copy the array or try to share the underlying storage.\n",
    "\n",
    "```C#\n",
    "// Never copy:\n",
    "public static Tensor from_array(Array input)\n",
    "\n",
    "// Copy only if dtype or device arguments require it:\n",
    "public static Tensor frombuffer(Array input, ScalarType dtype, long count = -1, long offset = 0, bool requiresGrad = false, Device? device = null)\n",
    "public static Tensor as_tensor(Array input,  ScalarType? dtype = null, Device? device = null)\n",
    "public static Tensor as_tensor(Tensor input, ScalarType? dtype = null, Device? device = null)\n",
    "\n",
    "// Always copy:\n",
    "public static Tensor as_tensor(IList<<VARIOUS TYPES>> input,  ScalarType? dtype = null, Device? device = null)\n",
    "public static Tensor tensor(<<VARIOUS TYPES>> input, torch.Device? device = null, bool requiresGrad = false)\n",
    "```\n",
    "\n",
    "In all cases, the resulting tensor gets its element type from the input array, unless it's overriden by the `dtype` argument. The tensor will be on the CPU unless overriden by the `device` argument.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.tensor(new float[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }).print();\n",
    "torch.tensor(new long[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }).print();"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since `torch.from_array` doesn't copy the data, modifying the tensor or the array will affect the other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "var arr = new float[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };\n",
    "var tArr = torch.from_array(arr);\n",
    "\n",
    "tArr.print();\n",
    "arr[4] = 100;\n",
    "tArr.print();\n",
    "tArr[6] = 200;\n",
    "arr"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Extracting an Array from a Tensor\n",
    "\n",
    "To get access to the internal data of a tensor as .NET array, use the 'data<T>()' property. It doesn't return an array, but a TensorAccessor<T>, which implements IEnumerable<T> and can therefore be turned into an array. The TensorAccessor<T> instance keeps a reference to the tensor data, but calling `ToArray<T>()` on it will create a copy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "var t = torch.rand(2,5);\n",
    "TorchSharp.Utils.TensorAccessor<float> ta = t.data<float>();\n",
    "float[] a = ta.ToArray();\n",
    "a"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ranges\n",
    "\n",
    "Another common and convenient tensor factory is `arange()`, which is used to create a 1D tensor with numbers ranging from a min to a max, exclusive of the max. You can provide the step value, or let it be the default, which is 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.arange(3,19)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, `arange` behaves a little differently from other factories -- the default element type is not float32, it's inferred from the arguments. To use floating-point, we an either specify the type, or use floating-point values as arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.arange(3.0f, 14.0f)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The step argument will allow us to get a more fine-grained number series:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.arange(3.0f, 5.0f, step: 0.1f)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## reshape()\n",
    "\n",
    "There is no way to make `arange` produce anything but a 1D tensor, so a common thing to do is to reshape it as soon as it's created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When printing out floating-point tensors, it is often helpful to provide a formatting string, applied only to FP elements, including the real and imaginary parts of complex numbers. `.ToString(true, fltFormat: \"0.00\")` would be a lot to type every time, so TorcSharp defines a shorter (and Python-looking) method `str()` that does the trick. `print()` also takes a format string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).str(fltFormat: \"0.00\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`reshape()` is, of course, useful for many other things, too, not just shaping the result of `arange()`. One thing that is very useful to know is that you can pass in '-1' for __one__ of the dimensions, and it has a very special meaning. Let's look at an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t = torch.rand(3,4,4,4);\n",
    "t.reshape(12, 4, 4).ToString()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t.reshape(-1,4,4).ToString()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t.reshape(4,-1,6).ToString()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, -1 is a wildcard. After the rest of the arguments specify their respective sizes, the -1 dimension is determined from the overall number of elements and the dimensions that have been specified. Obviously, it can only be used to construct a proper tensor if the other dimensions are correct. This, for example, results in an exception:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t.reshape(4,-1,5).ToString()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Common Tensor Properties and Operations\n",
    "\n",
    "There is a large number of operations on tensors. Here are some of the most commonly used ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "t = torch.arange(3.0f, 5.0f, step: 0.1f).reshape(2,2,5);\n",
    "\n",
    "// The overall shape of the tensor:\n",
    "t.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// The number of dimensions:\n",
    "t.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// The total number of elements held in the tensor:\n",
    "t.numel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Move the tensor to the GPU and back to the CPU:\n",
    "\n",
    "// t.cuda() // Uncomment if you're using one of the CUDA backends.\n",
    "t.cpu()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "polyglot_notebook": {
     "kernelName": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// On Apple HW (not x64), move to MPS and back:\n",
    "\n",
    "// t.mps().cpu() // Uncomment if you're running on moder Apple HW."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Getting the transpose of a matrix:\n",
    "torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).print();\n",
    "torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).T.print();"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's important to note that transposing isn't the same as simply reshaping to a different shape:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.arange(3.0f, 5.0f, step: 0.1f).reshape(5,4).print(fltFormat: \"0.00\");\n",
    "torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).T.print(fltFormat: \"0.00\");"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you need to have some idea of how many tensors you have allocated in you process, there's a static property `TotalCount` that tells you just how many .NET tensors are active. That's not all tensors, just those that have a .NET representation. Temporaries used by the native library and not surfaced to managed code don't count.\n",
    "\n",
    "The property is useful if you are diagnosing memory issues, for example in a training loop. If the number of tensors keeps growing, somewhere there's a missing Dispose() call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.Tensor.TotalCount"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's also a `PeakCount` static property that keeps track of the high-water mark:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.Tensor.PeakCount"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a copy of a tensor by calling `clone()`. The new tensor will have its own backing storage space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "// Clone a tensor:\n",
    "var s = t.clone();\n",
    "s[0,0,1] = torch.tensor(375);\n",
    "s.print();\n",
    "t"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a new managed code reference to the same underlying tensor data by calling `alias()`. This is useful when a function needs to return an input tensor, but the caller has reason to expect a fresh reference. For example, the caller may assume that you can call `Dispose()` on the resulting tensor reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "var a = t.alias();\n",
    "a[0,0,1] = torch.tensor(250);\n",
    "t"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TorchSharp has great support for complex tensors, which are as easy to create as others. Complex numbers are commonly used in signal processing scenarios, such as audio analysis. Complex numbers have two components -- the real and the imaginary parts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "var ct = torch.rand(3,4,dtype:torch.ScalarType.ComplexFloat32);\n",
    "ct.str(fltFormat:\"0.00\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the real and imaginary parts, there are property accessors for that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "ct.real"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "ct.imag"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you create random complex tensors, both the real and imaginary parts are random. However, when you create a tensor with `torch.ones`, only the real part is filled in -- the imaginary part is '0'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "polyglot-notebook"
    }
   },
   "outputs": [],
   "source": [
    "torch.ones(3,4,dtype:torch.ScalarType.ComplexFloat32)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Experienced PyTorch users will notice that TorchSharp doesn't represent complex numbers in the str() output exactly the same: TorchSharp uses 'i' instead of 'j' to indicate the imaginary part, and we don't print the real or imaginary part if it is zero (unless they are both zero, of course)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "name": "C#"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
